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Abstract:  Advances  in  automated  data  acquisition  in  electron 
tomography  have  led  to  an  explosion  in  the  amount  of  data  that  can 
be  obtained  about  the  spatial  architecture  of  a  variety  of 
biologically  and  medically  relevant  objects  with  sizes  in  the 
“nano”  range  of  lO-lOOOnm.  The  development  of  methods  to 
analyze  the  vast  amounts  of  information  contained  in  these 
tomograms  is  a  major  challenge  since  the  electron  tomograms  are 
intrinsically  noisy.  A  fundamental  step  in  the  automatic  analysis  of 
large  amounts  of  data  for  statistical  inference  is  to  segment  3D 
features  in  cellular  tomograms  that  can  work  robustly  and  rapidly 
despite  of  low  signal  to  noise  ratios  inherent  to  biological  electron 
microscopy.  This  work  evaluates  various  denoising  techniques  on 
tomograms  obtained  using  dual- axis  simultaneous  iterative 
reconstruction  (SIRT)  technique.  Using  three-dimensional  images 
of  HIV  in  infected  human  macrophages  as  an  example,  we 
demonstrate  that  transform  domain-denoising  techniques 
significantly  improve  the  fidelity  of  automated  feature  extraction. 
Importantly,  our  approaches  represent  an  vital  step  in  automating 
the  efficient  extraction  of  useful  information  from  large  datasets  in 
biological  tomography,  and  facilitate  the  overall  goal  of  speeding 
up  the  process  of  reducing  gigabyte-sized  tomograms  to  byte-sized 
data. 

Keywords:  Electron  tomography,  Denoising,  Feature  extraction, 
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1.  INTRODUCTION 

Electron  tomography  (ET)  is  an  emerging  tool  to  describe  the 
three-dimensional  (3D)  architectures  of  large  molecular 
complexes,  viruses  and  cells.  ET  is  a  general  method  for  three- 
dimensional  reconstruction  of  electron  transparent  objects  from  a 
series  of  projection  images  recorded  with  a  transmission  electron 
microscope.  At  high-doses,  electrons  interact  with  unstained 
organic  matter  which  leads  to  breaking  of  chemical  bonds  and  the 
creation  of  free  radicals  which  in  turn  cause  further  secondary 
damage.  At  low-doses  it  is  possible  to  scatter  of  electrons  from 
biological  matter  such  as  proteins  and  record  molecular  images, 
but,  at  the  expense  of  poor  signal-to-noise  ratio  (SNR)  [1].  The 
challenge  for  achieving  high  resolution  with  electron  imaging  of 
unstained  specimens  then  is  that  doses  that  are  high  enough  to  get 


Fig.  1.  Imaging  Pipeline. 

a  good  SNR  lead  to  unacceptable  specimen  damage,  while  doses 
that  are  low  enough  to  preserve  the  specimen,  lead  to  poor  image 
quality.  Hence  there  is  a  trade-off  between  the  dosage  level  and 
the  damage  caused  to  the  specimen.  The  use  of  stains  allows 
imaging  to  be  carried  out  using  higher  electron  doses,  although  this 
does  not  completely  solve  the  problem  of  low  SNR.  The  main 


steps  in  imaging  and  image  processing  are  highlighted  in  Fig. 
l.The  data  collection  includes  taking  a  series  of  images  at  different 
relative  tilts  of  beam  and  specimen  as  shown  in  Fig.  2.  The 
electron  microscope  provides  2D  projection  images.  Tomograms 
are  generated  by  computational  analysis  of  a  series  of  electron 
micrographs  taken  at  varying  tilt  angles,  usually  from  -80°  to  +80°. 
Using  algorithms  that  implement  weighted  back  projection  or 
SIRT,  one  can  obtain  3D  volumes  of  the  specimen  from  projection 
images  as  shown  in  Fig.  2. 


Fig.  2.  Data  Collection  procedure  (left)  and  reconstruction  using 
Weighted  Back  Projection  (WBP)  (right) 


As  techniques  for  data  collection  and  tomogram 
reconstruction  are  becoming  more  streamlined,  the  development  of 
methods  to  analyze  the  enormous  amounts  of  information  in  these 
tomograms  remains  a  major  challenge  [1].  Efforts  to  establish  tools 
for  quantitative  interpretation  of  tomograms  are  beginning  to  be 
applied  to  a  range  of  biological  problems,  as  reviewed  recently  [2]. 
A  long-term,  goal  of  this  type  of  electron  tomography  is  to 
interpret  the  spatial  arrangement  of  the  constituent  molecular 
components  at  high  resolution.  Typically,  the  identification  of 
simple  structural  features,  such  as  membrane  boundaries,  or 
protein  complexes,  such  as  ribosomes,  is  made  possible  by 
differentially  stained  cellular  components  relative  to  their 
immediate  neighbors.  Electron  tomograms,  are,  however, 
intrinsically  noisy,  and  this  poses  significant  challenges  for  image 
interpretation,  especially  in  the  context  of  high-throughput  data 
collection  and  analysis.  A  fundamental  step  in  the  automatic 
analysis  of  large  amounts  of  data  for  statistical  inference  is  to 
employ  robust  methods  for  3D  segmentation  that  can  work  well  at 
low  SNR.  The  target  of  these  strategies  is  to  improve  the  signal  as 
much  as  possible  relative  to  the  noise  level  to  mine  3D  information 
from  complex  tomograms. 

This  work  investigates  transform  domain  denoising  techniques 
in  the  context  of  electron  tomography  of  viruses.  Image  denoising 
approaches  have  been  extensively  investigated  previously  for 
various  applications  [3-7].  Our  primary  goal  here  is  to  evaluate  the 
performance  of  a  range  of  denoising  techniques  to  electron 
tomograms  recorded  at  room  and  cryogenic  temperatures  and  study 
the  implications  of  electron  doses  and  noise  on  denoising  of 
tomograms.  Denoising  techniques  boost  the  SNR,  facilitate  better 
feature  extraction,  and  enable  the  application  of  automated 
techniques.  For  analysis  and  feature  extraction,  we  have  used 
tomograms  of  HIV  and  the  related  virus,  SIV,  recorded  at  either 
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room  or  cryogenic  temperatures,  and  reconstructed  using  SIRT- 
based  approaches.  We  implemented  dual-axis  data  collection  in 
this  work  since  it  has  less  missing  information  in  the  tomogram, 
and  we  used  SIRT  because  of  the  improved  SNR  in  the 
reconstructed  tomograms  as  compared  to  conventional  back 
projection  methods.  We  report  here  on  the  relative  merits  of  a 
variety  of  denoising  algorithms  on  the  detection,  clustering  and  the 
spatial  distribution  of  spikes  in  SIV/HIV  viruses,  and  have 
compared  their  performance  using  single-image  SNR  estimation 
and  Fourier  ring  correlation  techniques.  Information  on  spike 
architecture  and  spacing  (variability)  generated  from  automated 
denoising  and  segmentation  could  be  fundamental  to  vaccine 
development.  The  performance  of  these  denoising  methods  was 
tested  on  a  variety  of  viral  and  cellular  tomograms  and  we  found 
that  the  phase  preserving  algorithm  (described  in  section  3) 
significantly  improved  the  SNR  while  retaining  the  same  overall 
architectural  information  in  the  tomograms.  We  validate  our 
approach  by  comparison  of  semi-automated  and  automated 
segmentation  procedures  on  tomograms  recorded  at  both  room  and 
cryogenic  temperature. 

2.  DATA  COLLECTION  AND  RECONSTRUCTION 
TECHNIQUES 

In  ET,  the  range  of  tilt  angles  is  often  limited  to  ±80°  due  to 
the  physical  constraints  imposed  by  the  microscope,  sample  and 
holder  which  leads  to  a  ‘missing  wedge’  of  information.  This  loss 
of  information  results  as  artifacts  in  the  reconstruction  including  an 
elongation  of  the  reconstruction  in  the  direction  of  the  optic  axis, 
perpendicular  to  the  tilt  axis.  All  reconstructed  volumes  contain  a 
significant  level  of  noise  that  arises  in  part  from  the  inherent 
limitations  of  current  electron  optics,  such  as  chromatic  and 
spherical  aberration,  beam  incoherence,  point  spread,  and  certain 
terms  of  the  lens  transfer  function,  such  as  contrast  transfer  and 
inelastic  scattering  (although  the  latter  can  be  reduced  by  use  of  an 
energy  filter).  Other  sources  of  noise  arise  from  the  CCD  detector; 
and  in  part  from  computational  manipulation  of  the  data,  including 
inaccuracies  in  alignment  of  the  tilt  series  and  reconstruction  of 
objects  that  have  missing  information  in  Fourier  space.  In  the 
special  case  of  cryo  electron  tomography  the  necessity  of  using  low 
electron  doses  is  an  additional  factor  contributing  to  noise.  The  use 
of  dual  axis  tomography  in  which  two  tilt  series  are  recorded  with 
mutually  perpendicular  tilt  axes,  reduces  the  missing  wedge  into  a 
‘missing  pyramid’  [8].  We  implement  an  algorithm  based  on  SIRT 
that  is  performed  by  comparing  at  each  iteration,  re-projections  of 
a  reconstruction  with  each  projection  (image)  obtained  in  the 
microscope,  using  the  original  projection  series  as  a  constraint  on 
the  output  of  the  next  iteration  [9].  This  procedure  is  known  to 
reduce  the  reconstruction  artifacts  starting  from  noisy  projection 
images. 

Room  temperature  Tomograms:  Tilt  series  of  90-nm-thick 
sections  of  fixed,  stained,  plastic-embedded  HIV-infected 
monocyte-derived  macrophages  were  collected  through  a  range  of 
±  80°,  at  2°  intervals  from  0°  to  ±50°  and  1°  intervals  from  ±50°  to 
±80°,  on  a  Tecnai  12  Transmission  Electron  Microscope  (FEI, 
Netherlands),  at  1-pm  defocus.  The  sample  was  rotated  90°  and  an 
orthogonal  tilt  series  of  the  same  area  was  collected.  Prior  to 
acquisition,  both  surfaces  of  the  sections  were  coated  with  15-nm 
gold  fiducial  markers  for  alignment.  Images  were  recorded  on  a 
4K  X  4K  Gatan  CCD  camera  binned  to  2K  by  2K,  at  a 
magnification  that  produces  a  pixel  size  corresponding  to  0.44  nm 
at  the  specimen  level.  Acquisition  was  automated  using  the 
XploreSD  software  (FEI,  Netherlands).  Each  tilt  series  was  aligned 


using  the  IMOD  package  [10].  The  aligned  series  were  then 
reconstructed  in  either  IMOD,  using  weighted  back  projection 
(WBP),  or  InspectSD  (FEI,  Netherlands),  using  Simultaneous 
Iterative  Reconstruction  Technique  (SIRT)  with  17  iterations,  to 
obtain  orthogonal  tomograms.  The  orthogonal  WBP  tomograms 
were  combined  in  IMOD  to  obtain  the  final  dual-axis  WBP 
tomogram,  and  the  orthogonal  SIRT  tomograms  were  combined  in 
IMOD  to  obtain  the  final  dual-axis  SIRT  tomogram..  By 
comparison,  single-axis  reconstructed  volumes  contain  a  relatively 
large  missing  wedge  of  information  in  Fourier  space,  resulting  in 
anisotropic  resolution  and  greater  distortion  of  the  3-D  structures 
of  the  reconstructed  objects. 

Cryo  tomograms:  We  used  cryo-electron  tomography  to 
analyze  the  three-dimensional  architecture  of  individual  SIV 
particles  under  near-native  conditions.  Preparations  of  the  gp41- 
truncated  variant  of  the  Mac239  SIV  strain  were  rapidly  vitrified 
by  plunge-freezing  into  liquid  ethane  cooled  to  —  180°C,  and 
imaged  at  6  micron  defocus  at  liquid  nitrogen  temperatures  using  a 
300  kV  electron  microscope  equipped  with  a  field  emission  gun 
and  an  energy  filter.  The  tilt  series  acquisition  was  done  collecting 
an  image  every  3°  intervals  from  ±65°.  Since  these  are  from 
unstained  virus  preparations,  the  contrast  in  the  images  arises  from 
the  intrinsic  density  of  the  virus.  The  spikes  evident  on  the  surface 
of  the  particles  are  ~  8-10  nm  wide  and  ~12  nm  high,  consistent 
with  the  dimensions  expected  from  a  trimer  of  gpl20  molecules 
protruding  from  the  surface  of  the  virion.  Projection 
images  of  the  particles  reveal  the  characteristic  distribution  of  the 
putative  gpl20-gp41  viral  spikes  on  the  surface  of  the 
virion  and  a  glimpse  of  the  internal  core  of  the  virus. 

3.  DENOISING  METHODS  AND  SIMULATION 
RESULTS 


In  this  section  we  provide  a  brief  explanation  of  the 
denoising  techniques  used  to  analyse  the  tomographic  data.  Our 
main  goal  is  to  apply  the  existing  denoising  techniques  to  both 
room  and  cryogenic  temperature  3D  tomograms  and  study  their 
performance  at  various  dosage  levels  and  noise  conditions.  This  is 
particularly  of  great  interest  since  one  denoising  algorithm  will  not 
perform  the  best  on  diverse  datasets.  The  tomograms  also  need  to 
be  qualitatively  examined  by  a  biological  expert  to  determine 
whether  the  features  of  interest  are  well  preserved  after  denoising. 
Thus  we  used  both  qualitative  and  quantitative  measures  to  choose 
the  optimal  denoising  algorithm.  We  implement  and  compare  the 
performance  of  anisotropic  diffusion  [3]  and  complex  diffusion 
algorithms  [4]  on  electron  tomograms.  The  equations  for 
anisotropic  diffusion  in  3D  are  given  as/^  =  V.(c(x,y,z,0V/) , 


where  k  is  the  conductance  parameter. 


c{x,y,z,t)  =  1/(1  ± 


-)  and  c{x,y,z,t)  =  e 


2/F 


(1) 


and  /  =  I{x,y,z,t)  is  the  3D  tomogram.  For  the  complex  diffusion. 


=  V.(c(Im(/))V/)  with  c(Im(/))  =  / 


1± 


^Im(/)Y 
V  ke  , 
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where  Im(/)  is  the  imaginary  value,  6  is  the  phase  angle  and  k  is 
the  threshold  parameter.  Another  method  of  denosing  is  to 
preserve  the  phase  data  [5]  by  applying  the  continuous  wavelet 
transform  and  we  use  log  Gabor  functions  to  construct 
symmetric/anti- symmetric  wavelet  filters.  The  process  determines  a 
noise  threshold  at  each  scale  and  shrinking  the  magnitudes  of  the 
filter  response  vectors,  the  phase  information  can  be  preserved 
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after  reconstruction.  The  other  denoising  methods  we  used  for 
comparison  are  based  on  translation  invariant  wavelets  with  soft 
thresholding  [6],  Bayes  least-squares  Gaussian  scale  mixtures 
(BLS-GSM)  [7]  where  we  used  steerable  pyramid  wavelets  and 
Weiner  filtering.  We  used  Wavelab  and  BLS-GSM  MATLAB 
software  available  from  the  authors’  website  for  both  these 
methods  with  modifications.  For  all  these  methods  we  obtained  the 
highest  SNR  by  varying  the  parameters  over  a  valid  range  and  also 
making  sure  that  the  spatial  features  of  interest  are  well  retained. 

Next,  we  present  segmentation  results  on  single/dual-axis  and 
cryo  tomograms  facilitated  by  3D  denoising  of  tomograms.  The 
effect  of  denoising  is  clearly  evident  from  Fig.  3(c)&(d)  and  Fig. 
4(a-d)  where  we  can  observe  a  surface  with  a  complete  membrane, 
distinct  core,  and  envelope  protein  spikes.  The  quantitative 
analysis  procedures  such  as  peak-SNR  (PSNR)  and  \ImSE 
assume  that  a  clean  image  is  available  for  analysis.  For  quantitative 
analysis  of  the  denosing  methods  we  implement  single-image  SNR 
estimation  [11]  since  we  are  working  with  tomograms  which  are 
essentially  stacks  of  2D  slices  of  the  reconstructed  volume.  We 
compute  the  SNR  of  each  slice  using  various  denoising  techniques 
and  the  result  shown  in  Fig.  3(c)  is  an  average  SNR  over  10 
consecutive  slices.  The  single-image  SNR  estimation  method 
assumes  that  the  noise  is  white  and  is  uncorrelated  from  pixel  to 
pixel.  The  SNR  is  given  as  [1 1] 

SNR=^=  Cim-Ml  am- Ml 

(i-A,)  af-(C(o,o)-^f)  ^„(o,o)-C(o,o)' ^ 

where  ^^^(0,0) ,  are  the  autocorrelation  function  (ACF) 

of  the  noisy  image,  noise  free  (NF)  image  and  p  is  the  correlation 
coefficient,  and  are  the  mean  and  variance  of  the  noisy 
image  respectively.  Since  we  do  not  have  the  noise  free  image, 
0j^^(O,O)  is  estimated  by  assuming  that  its  value  is  same  as  the 
ACF  of  the  neighboring  offsets  as, 
(0,0)^  ~  (0ii(l,O) -h  ^jj(0,l))/2  .  This  estimate  is  reasonable  if 

the  ACF  changes  slowly  at  the  origin  which  is  the  case  for  room 
temperature  and  cryo  tomograms.  Single  Image  SNR  estimation  on 
a  single  axis/dual- axis  SIRT  reconstructed  and  cryo  tomogram  in 
Fig.  3(a)  depicts  that  the  anisotropic  diffusion  algorithm  performs 
the  best  in  terms  of  the  SNR,  but  the  image  features  are  lost  in  the 
denoising  process  unlike  the  phase  preserving  (PP)  method.  For 
e.g.,  the  SNR  of  the  dual-axis  SIRT  noisy  tomogram  was  boosted 
significantly  from  S.IdB  as  shown  in  Fig.  3(a).  Furthermore,  we 
also  compute  the  Fourier  ring  correlation  (FRC)  [12]  to  compare 
various  reconstruction  and  denoising  methods.  From  Fig.  3(b)  we 
can  observe  that  the  PP  method  on  a  dual-axis  SIRT  reconstruction 
performed  better  than  other  approaches. 

Segmentation  procedure:  Tomograms  were  segmented  using 
the  Amira  Visualization  software  [13]  which  executes  a  procedure 
of  3D  region-growing  in  which  the  largest  connected  area  that 
contains  the  user-selected  voxel  and  all  voxels  within  a  user- 
defined  range  of  gray  values  is  selected  and  can  be  used  to 
generate  a  volume.  Our  threshold  was  based  on  a  range  relative  to 
the  seed  voxel  selected,  although  other  methods  using  absolute 
scales  for  contrast  can  also  be  used.  This  procedure  thus  allows  for 
“one-click”  segmentation  of  the  tomogram.  Identical  points  in  the 
center  of  each  virion  were  selected  to  initiate  the  3D  growing 
process  and  create  a  surface.  We  also  carried  out  semi-automated 
segmentation  in  which  user-designated  regions  of  the  tomogram 
are  subjected  to  a  threshold  (Fig.  4(a-d)).  We  then  compare  the 
results  of  the  one-click  segmentation  for  both  single  and  dual-axis 
SIRT  reconstructed  HIV  virion  that  was  imaged  at  room 


temperature  against  semi- automated  segmentation  (second  row  of 
Fig.  4).  The  one-click  segmentation  of  the  denoised  SA-SIRT 
tomogram  gave  a  surface  that  resembled  a  virion  with  a  core  and  a 
few  envelope  protein  spikes  (Fig.  4(f)).  One-click  segmentation  of 
a  non-denoised  DA-SIRT  tomogram  returned  a  surface  that  was 
highly  non-contiguous  and  included  noise  (Fig.  4(g)).  After  the 
tomogram  was  denoised,  one-click  segmentation  yielded  a  surface 
with  a  complete  membrane,  distinct  core,  and  envelope  protein 
spikes  that  closely  matched  the  location  of  the  spikes  from  semi- 
automatically  segmented  version  of  the  DA-SIRT  denoised 
tomogram  as  in  Fig.  4(h).  For  the  single  axis  tomograms  obtained 
from  data  recorded  at  cryogenic  temperatures  (last  row.  Fig.  4(i-l)) 
only  semi-automated  segmentation  is  possible  due  to  the  higher 
noise  levels.  The  results  in  the  denoised  tomogram  shown  in  Fig. 
4(1)  depict  that  the  spikes  are  well  separated  and  the  spike-volume 
is  more  distinct.  In  Table  1,  we  present  a  comparison  of  the  fully 
and  semi- automated  approaches  on  the  DA-SIRT  reconstructed 
tomogram  in  terms  of  the  volume  estimates  of  individual  viral 
spikes.  The  variation  between  these  two  approaches  is  within  25%. 


Voxel  size  (Semi- 
Automated) 

Voxel  size  (Automated) 

Relative  Error 

1 

942.6560 

825.36 

0.1244 

2 

757.5100 

678.92 

0.1037 

3 

962.9190 

1209.40 

0.2560 

4 

849.7470 

691.77 

0.1859 

Table  1.  Volume  occupied  by  the  viral  spikes. 

4.  CONCLUSION  AND  FUTURE  WORK 


Electron  tomograms  are  intrinsically  noisy  and  this  poses 
significant  challenges  for  image  interpretation,  especially  in  the 
context  of  low  dose  and  high-throughput  data  analysis.  To 
maximize  the  information  in  reconstructed  tomograms,  we  have 
used  dual  axis  data  collection  strategies,  and  to  obtain  the  highest 
starting  SNR,  we  have  used  SIRT-based  reconstruction 
approaches.  Our  goal  has  been  to  evaluate  the  relative  performance 
of  different  denoising  methods  in  further  improving  the  SNR,  and 
to  test  whether  these  denoised  tomograms  can  be  processed 
automatically  to  extract  biologically  relevant  information.  We 
demonstrate  that  denoising  significantly  improves  the  fidelity  of 
automated  feature  extraction,  and  that  phase  preserving  algorithm 
performs  best  for  recovering  structural  information.  We  show  that 
molecular  information  such  as  the  average  volume  of  individual 
viral  spikes  can  be  obtained  from  these  denoised  tomograms,  and 
that  the  values  closely  match  those  obtained  using  manual  user- 
guided  approaches.  This  tool  thus  represents  an  important  element 
in  the  overall  pipeline  for  HIV  imaging  for  analyzing  the  structural 
origins  of  variation  in  different  viral  isolates. 
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Fig.  3.  (a)  Single-image  SNR  for  various  denoising  methods  for  room  temperature  and  cryo  tomograms,  (WF-Weiner  Filter,  TI/ST-Translation 
invariant  with  soft  thresholding,  PMl  and  PM2-  Perona-Malik  equations  1&2,  NLD-Nonlinear  Diffusion,  CD-Complex  Diffusion,  PP-Phase 
preserving  and  BGB-  BLS-GSM  method,  (b)  Fourier  Ring  Correlation  (FRC)  for  Dual- Axis  and  Single- Axis  reconstructed  HIV  tomograms  using 
various  denoising  methods.  Semi-automted  segmentation  of  Dual-axis  SIRT  reconstructed  HIV  tomogram  (c)  noisy  &  (d)  denoised. 


Fig.  4.  Top  Row  (a-d):  Semi-Automated  Tilted.  Second  Row  (e-h):  Comparison  of  the  Semi-Automated  and  Automated  segmentation  results.  (Left 
two  figures  in  each  row)  SA-SIRT  reconstructed  HIV  virion  &  (Right  two  figures  in  each  row)  DA-SIRT  reconstructed  HIV  virion.  Last  Row  (i- 
1):  Cryo  SIV  virion  (Left),  Cryo  SIV  spike  spatial  distribution  (Right).  (ND-Non  Denoised,  D-Denoised,  SA-Single  axis  &  DA-  Dual  Axis). 
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